An Introduction to Adaptive Critic Control: A Paradigm Based on Approximate Dynamic Programming

Adaptive critic control is an advanced control technology developed for nonlinear dynamical systems in recent years. It is based on the idea of approximate dynamic programming. Dynamic programming was introduced by Bellman in the 1950’s for solving optimal control problems of nonlinear dynamical systems. Due to its high computational complexity, applications of dynamic programming have been limited to simple and small problems. The key step in finding approximate solutions to dynamic programming is to estimate the cost function in dynamic programming. The optimal control signal can then be determined by minimizing (or maximizing) the cost function. Due to their universal approximation capability, artificial neural networks are often used to represent the cost function in dynamic programming. The implementation of approximate dynamic programming usually requires the use of three modules-Critic, Model, and Action. These three modules perform the function of evaluation, prediction, and decision, respectively. This article introduces some basic algorithms of adaptive critic control and some recent development of the area. It will also outline some future perspectives of this new control technology.